HubClient: retry transient Hub tree-listing failures (429/5xx)#2
Open
AurionRodgerDiablo wants to merge 1 commit into
Open
HubClient: retry transient Hub tree-listing failures (429/5xx)#2AurionRodgerDiablo wants to merge 1 commit into
AurionRodgerDiablo wants to merge 1 commit into
Conversation
listFiles treated any non-200 from the Hub tree API as variantNotFound,
surfacing "No '<path>' variant in <repo>@<rev> — this model may not be
published for this platform" even when the subtree exists. The Hub sheds
load with transient 429s ("maximum queue size reached") and occasional
5xx; a single 429 on any of a multi-bundle model's subtree listings (OCR
resolves vision/decoder/assets/tokenizer) failed the whole download with
a misleading message. The large-file download path already retried 6x;
only the tree listing did not.
Retry the listing up to 5x with exponential backoff (2/4/8/16s) on 429/5xx.
A firm 4xx (e.g. 404) still means the subtree really isn't published, so it
keeps variantNotFound. Exhausting retries throws httpError(statusCode:) with
the real code instead of the misleading "not published" message.
Reproduced and verified fixed with mlboydaisuke/Unlimited-OCR-CoreAI via
Examples/ReadDoc (swift run readdoc-cli --model unlimited-ocr).
Claude-Session: https://claude.ai/code/session_01HyaY1ZCanqUvYbWVD4Rc53
There was a problem hiding this comment.
Pull request overview
This PR improves robustness of Hugging Face Hub subtree listing by retrying transient Hub tree API failures, preventing misleading variantNotFound errors when the subtree exists but the API is temporarily shedding load.
Changes:
- Add retry loop with exponential backoff for Hub tree listing requests.
- Treat non-429 4xx responses as
variantNotFound, but surface exhausted retries ashttpError(statusCode:).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+41
to
+43
| if attempt > 0 { | ||
| try await Task.sleep(nanoseconds: UInt64(1 << attempt) * 1_000_000_000) | ||
| } |
Comment on lines
+55
to
57
| guard status == 200 else { | ||
| throw CoreAIKitError.httpError(statusCode: status, file: api.absoluteString) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
HubClient.listFilestreats any non-200 response from the Hub tree API asvariantNotFound, surfacing:…even when the subtree exists. In practice the Hub sheds load with transient
429 {"error":"maximum queue size reached"}(and occasional 5xx). Multi-bundle models list several subtrees (OCR resolvesvision/decoder/assets/tokenizer) and those calls race, so a single 429 aborts the whole download with a misleading "not published" message.Reproduced with
mlboydaisuke/Unlimited-OCR-CoreAIviaExamples/ReadDoc: hitting the tree API for the four subtrees returns 429 on some of them, and the run fails even though every subtree is present onmain.The large-file download path already retries 6×; only the tree-listing step did not.
Fix
Retry the tree listing up to 5× with exponential backoff (2/4/8/16 s) on 429/5xx. A firm 4xx (e.g. 404) still means the subtree really isn't published, so it keeps
variantNotFound. Exhausting retries throwshttpError(statusCode:)with the real status instead of the misleading message.One file,
Sources/CoreAIKitCore/HubClient.swift(+22/-3). No API or behavior change on the success path.Verification
swift run readdoc-cli --model unlimited-ocr --image sample.pngnow completes (exit 0) through transient 429s and prints the document markdown.https://claude.ai/code/session_01HyaY1ZCanqUvYbWVD4Rc53